Use of Meta-data for Value-level Inconsistency Detection and Resolution During Data Integration

This paper addresses the data integration problem: there exists a collection of autonomous heterogeneous information sources that need to be integrated; users want to be able to query the collection transparently and to get a single, unambiguous answer. The sources may conflict with each other on three levels: their schemas, data representation, or data themselves. One has to resolve the conflicts that may arise during the integration to get the single answer to a query. Most of the approaches in this area of research resolve inconsistencies among different schemas and data representations, and ignore the possibility of data value-level conflict altogether. The few that do acknowledge its existence are mostly probabilistic approaches which just detect the conflict and provide a user with some additional information on the nature of the inconsistency (e.g. give a set of conflicting values with attached probabilities). We propose an extension to the relational data model that makes use of meta-data of the information sources called properties. This extension gives ground to a flexible data integration technique described in this paper that consists of three phases: (1) query result construction, (2) data conflict detection and (3) data conflict resolution. An improvement to data clustering techniques in the conflict detection phase is presented in the paper. It uses another type of meta-data available from the sources (source descriptions in terms of the virtual database schema) to narrow down the areas of possible data conflicts. For the last phase, a flexible data-level conflict resolution algorithm is offered, which incorporates both contentand property-based approaches. The algorithm is guided by user-defined priorities of properties and by domain-based resolution strategies.